Fixing static builds is great, but this doesn't seem like the right way to do it.

• NIX_LDFLAGS is a hack that should only be used as last resort. It sneaks flags past the package's build system, thus often making downstream packages not work as expected.
• pkg-config should be able to figure that out, if used:

  % nix-shell -A openssh
  
  $ pkg-config --libs libedit
  -L/nix/store/sbfj9lhpfi5v9vwrdf9my8l1m7cwbi37-libedit-20191231-3.1/lib -ledit
  
  $ pkg-config --libs libedit --static
  -L/nix/store/sbfj9lhpfi5v9vwrdf9my8l1m7cwbi37-libedit-20191231-3.1/lib -ledit -lncurses

Notice how when --static is passed to pkg-config, it actually emits the required flags of the recursive dependencies.

The way that works is this:

$ echo $PKG_CONFIG_PATH | tr ':' '\n' | grep libedit | head -n1
/nix/store/wx8xws36cz66gabl9sapgaz9p70h7x1m-libedit-20191231-3.1-dev/lib/pkgconfig

$ cat /nix/store/wx8xws36cz66gabl9sapgaz9p70h7x1m-libedit-20191231-3.1-dev/lib/pkgconfig/libedit.pc
...
Libs: -L${libdir} -ledit
Libs.private: -lncurses 
...

Here we can see that the .pc file for libedit "remembers" its non-dynamic dependencies in Libs.private, so that when asked to be linked in statically by a downstream package, it can emit them.

This is pkg-config's direct way to address your observation:

It seems that this is symptom of more general problem: if library "foo"
depends on library "bar", then to link dynamically program that uses
library "foo" it is enough to pass "-lfoo" to compiler, but it is
necessary to pass "-lfoo -lbar" (in that order) to compiler to link it
statically.

So, the right way that this should work is that you tell openssh's build system to link itself statically, based on that it should invoke pkg-config with the --static flag, and then this manual addition of flags should not be necessary.

Clearly somewhere that process breaks down, and the best way forward is to investigate where.

That's a really great explanation, I learned something new today about how pkg-config works. As openssh maintainer I would agree the current state of the PR doesn't look like the right way to solve things but no idea what the right approach is so thanks for stepping in and posting @nh2!

@nh2 Thank you for review.

Unfortunately, openssh build system does not understand concept of static build, so I had to fall back on sed. Libedit was quite simple, since it used pkg-config, so I just replaced all calls to pkg-config with pkg-config --static.

Kerberos was harder. Despite library providing pkg-config files, build system uses either own shell code to figure out compiler flags. I added "kerberos.dev" into build inputs, so configure script control flow into the branch which is simpler to patch.

@KAction Yes, that looks good, great!

Please add what you described here as a comment on top of your seds in the nix file though, so that we keep the rationale next to the code.

Your change here might actually help me with nh2/static-haskell-nix#68 which is nice.

@KAction with the latest force-push

KAction force-pushed the KAction:openssh branch from 4879ea9 to 56a01c2 22 hours ago

you're moving back to the NIX_LDFLAGS approach -- just checking that it's intended.

I saw the updated commit message:

openssh: fix static, with-kerberos build

Error messages without this patch are quite misleading: they are
complaining about compilation where root of the problem is configure
stage.

Without these extra linking flags (mostly extracted from pkg-config
database), configure flag interprets undefined reference as missing
header/function error, resulting in problems at compile time.

so probably it's intended, and I think that's fine if there is no other way, but I'd definitely put that into a comment that explains that that is the reason why we're using NIX_LDFLAGS here for now, and also the concrete invocation that made you end up with those specific -l flags, so that we know how to update it.

Also, if you've already investigated that topic/failure a bit further, it would be great to post that info here.

Oops, pushed wrong branch. No, it is possible to get things working without NIX_LDFLAGS.